Overview:

We are looking for a Databricks Platform Operations Engineer to take charge of our cloud data infrastructure, ensuring its stability, performance, and security. This role is perfect for professionals with a strong background in big data operations, cloud technologies, and governance frameworks.

You’ll be the primary go-to person for data load monitoring, platform optimization, capacity planning, and governance, playing a crucial role in enabling data teams and ensuring a seamless and scalable environment.

Shift & Work Mode:

• Shift: 24x7 (rotational shift patterns may apply)
• Mode: Work From Office (Hyderabad or Bangalore locations)

Key Responsibilities:

1. Data Load Monitoring & Resolution

• Monitor ingestion dashboards to detect and resolve data pipeline issues in real-time
• Troubleshoot failures in data pipelines, working alongside engineering teams
• Ensure data availability and integrity through active monitoring and reporting
• Maintain detailed incident logs and resolution reports

2. Platform Optimization & Capacity Planning

• Continuously tune Databricks clusters for performance and cost efficiency
• Plan infrastructure scale-ups or optimizations based on workload analysis
• Recommend platform enhancements to support expanding data needs

3. Governance & Security Management

• Enforce data governance policies, including access control and data lineage
• Manage user permissions through Azure AD, AWS IAM, or similar tools
• Ensure compliance with organizational and regulatory data policies

4. Technical Architecture & Collaboration

• Act as a Databricks platform architect, advising on best practices
• Collaborate with data scientists, engineers, and governance teams
• Create and maintain technical documentation for architecture and procedures

5. Operational Excellence & Automation

• Use tools like Python, Bash, PowerShell, and Terraform to automate tasks
• Troubleshoot complex job failures and platform issues, including escalation
• Participate in on-call rotations for incident handling and recovery

Key Technical Skills:

Databricks, Azure, AWS, GCP, Jenkins Scripting, Pipeline Monitoring, Platform Optimization, Python, Bash, PowerShell, Azure AD, AWS IAM, Terraform, ARM Templates, Linux Administration, Hadoop, Apache Spark, Data Governance, RBAC, Data Lineage, SQL

Requirements:

Required Qualifications:

• Bachelor’s Degree in Computer Science, IT, or a related field
• 2+ years of experience in Databricks or similar cloud-based data platforms
• Strong operational skills in pipeline monitoring and issue resolution
• Hands-on experience in platform architecture, governance, and security
• Prior experience as a Linux/Hadoop/Spark Administrator is a huge advantage
• Proficient with cloud platforms and related services (Azure, AWS, or GCP)
• Good knowledge of identity and access management tools
• Expertise in automation using scripting languages
• Excellent communication and collaboration capabilities

Preferred Skills:

• Experience with infrastructure-as-code tools like Terraform or ARM templates
• Exposure to data governance tools like Azure Purview
• Familiarity with Apache Spark and SQL for platform support
• Understanding of data lakehouse architectures and best practices

Additional Perks & Benefits:

• Opportunity to work in one of the fastest-growing areas in cloud data engineering
• Hands-on exposure to modern data lakehouse and governance solutions
• Collaborative environment with a focus on platform reliability and automation
• Ideal for engineers looking to advance their cloud operations and DevOps skills

If you are passionate about cloud data platforms and want to build scalable and secure data infrastructure, this is a great opportunity to be at the center of digital transformation.